Approximate Strategic Reasoning through Hierarchical Reduction of Large Symmetric Games

To deal with exponential growth in the size of a game with the number of agents, we propose an approximation based on a hierarchy of reduced games. The reduced game achieves savings by restricting the number of agents playing any strategy to fixed multiples. We validate the idea through experiments on randomly generated local-effect games. An extended application to strategic reasoning about a complex trading scenario motivates the approach, and demonstrates methods for game-theoretic reasoning over incompletely-specified games at multiple levels of granularity. Motivation Consider the task of selecting among a large set of strategies to play in an 8-player game. Through careful judgment you manage to narrow down the candidates to a reasonable number of strategies (say 35). Because the performance of a strategy for one agent depends on the strategies of the other seven, you wish to undertake a game-theoretic analysis of the situation. Determining the payoff for a particular strategy profile is expensive, however, as your observations of prior game instances are quite limited, and the only operational description of the game is in the form of a simulator that takes a non-negligible time (say 10 minutes) to produce one outcome. Moreover, since the environment is stochastic, numerous samples (say 12) are required to produce a reliable estimate for even one profile. At two hours per profile, exhaustively exploring profile space will require 2 · 35 or 4.5 trillion hours simply to estimate the payoff function representing the game under analysis. If the game is symmetric, you can exploit that fact to reduce the number of distinct profiles to ( 42 8 ) , which will require 236 million hours. That is quite a bit less, but still much more time than you have. This is the situation we face as entrants in the annual Trading Agent Competition (TAC) travel-shopping market game (Wellman et al. 2003). The necessity of empirical evaluation in this setting combined with the infeasibility of exhaustive analysis prompts us to seek principled ways to direct a nonexhaustive exploration. In this paper we investigate the exploitation of hierarchical structure in the space of profiles to balance the goals of spanning the overall space and focusing effort on the most promising regions. Copyright c © 2005, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. The idea is that although a strategy’s payoff does depend on the play of other agents (otherwise we are not in a game situation at all), it may be relatively insensitive to the exact numbers of other agents playing particular strategies. For example, let (s,m; s) denote a profile where m other agents play strategy s, and the rest play s. In many natural games, the payoff for playing any particular strategy against this profile will vary smoothly with m. If such is the case, we sacrifice relatively little fidelity by restricting attention to subsets of profiles, for instance those with only even numbers of any particular strategy. To do so essentially transforms the N -player game to an N/2-player game over the same strategy set, where the payoffs to a profile in the reduced game are simply those from the original game where each strategy in the reduced profile is played twice. The potential savings from reduced games are considerable, as they contain combinatorially fewer profiles. The 4-player approximation to the TAC game (with 35 strategies) comprises 73,815 distinct profiles, compared with 118 million for the original 8-player game. In case exhaustive consideration of the 4-player game is still infeasible, we can approximate further by a corresponding 2-player game, which has only 630 profiles. Approximating by a 1-player game is tantamount to ignoring strategic effects, considering only the 35 “profiles” where the strategies are played against themselves. In general, an N -player symmetric game with S strategies includes ( N+S−1 N ) distinct profiles. Figure 1 10 20 30 40 50 60 1x100 1x101 1x102 1x103 1x104 1x105 1x106 1x107 1x108 1x109 1x1010